ffmpeg + other stuff! [be gentle]

[deeppurple1]

Ok, Am completely new to this and i need stuff explaining simply [stroke survivor]

In the Summer, Magentamusik [dot]de will be streaming a big Rock music festival

I'd like to download the stream BUT NOT as a screen capture..I can grab their offline shows using jdownloader2 with chrome behind a VPN set to Germany

will i be able to do this with a livestream or do i need ffmpeg commands etc on my win10 pc as i do not know how to do this please?, any help would be grateful

[lomero]

please provide a valid url to play

[white_snake]

Just grab the playlist url with The Stream Detector browser extension and then feed it to your favorite playlist downloader, be it yt-dlp, N_m3u8DL-RE or streamlink (the latter is the one best suited for live video, supposing it's not encrypted). Try this method with the offline shows to get familiar with it so that you'll know what to do when the live stream will be on.

[deeppurple1]

please provide a valid url to play

Try this as an offline thing https://www.magentamusik.de/iron-maiden-woa-konzert-vod

[deeppurple1]

Just grab the playlist url with The Stream Detector browser extension and then feed it to your favorite playlist downloader, be it yt-dlp, N_m3u8DL-RE or streamlink (the latter is the one best suited for live video, supposing it's not encrypted). Try this method with the offline shows to get familiar with it so that you'll know what to do when the live stream will be on.

I said i really have no clue, i have The Stream Detector..Putting something into ffmpeg and i get this [see attachment]..Is this correct?..This is from a different site than the one i want to use. That site does not do anything on already available items

[IceM]

Use N_m3u8DL-RE_M3

https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

[deeppurple1]

Use N_m3u8DL-RE_M3

https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

Thanks but as i said, I have no idea what this is or what to do

[IceM]

Install https://www.videohelp.com/software/N-m3u8DL-RE

and download your video example with this command

N-m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

[white_snake]

Use N_m3u8DL-RE_M3

https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

Thanks but as i said, I have no idea what this is or what to do

That is a command line. If you know how to use the ffmpeg cli, then you know how to use yt-dlp or N_m3u8DL-RE.

Install https://www.videohelp.com/software/N-m3u8DL-RE

and download your video example with this command

N-m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

That actually yields a very poor quality video. This is the command for the 1080p:

N_m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/420/index.m3u8

Unfortunately for OP, this .m3u8 is not grabbed successfully by The Stream Detector. You have to do some digging inside your dev tools to get it. The .m3u8 grabbed by the extension fails to download a proper file. There's some trap set somewhere (haven't checked) that makes downloaders produce a bad file.

[lomero]

@ IceM
what version is N_m3u8DL-RE_M3 ? and where have you find version 20230412 from your shot on post#6 ? latest workflow is 20231113 or stabbedbybrick pre rel version shared here

@ deeppurple1
if you know (and use) strem detector you can get also ffmpeg command for your video. or does not work for live video ?

[IceM]

@ IceM
what version is N_m3u8DL-RE_M3 ? and where have you find version 20230412 from your shot on post#6 ? latest workflow is 20231113 or stabbedbybrick pre rel version shared here

@lomero : typing error, sorry

[deeppurple1]

Use N_m3u8DL-RE_M3

https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

Thanks but as i said, I have no idea what this is or what to do

That is a command line. If you know how to use the ffmpeg cli, then you know how to use yt-dlp or N_m3u8DL-RE.

Install https://www.videohelp.com/software/N-m3u8DL-RE

Ok, Thanks firstly..I've managed to get the item, but how did you find the address "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/420/index.m3u8"

As i said, i really need things explaining very easily because of my memory issues


and download your video example with this command

N-m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/9000/index.m3u8 --save-name IronMaiden.mp4

That actually yields a very poor quality video. This is the command for the 1080p:

N_m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/420/index.m3u8

Unfortunately for OP, this .m3u8 is not grabbed successfully by The Stream Detector. You have to do some digging inside your dev tools to get it. The .m3u8 grabbed by the extension fails to download a proper file. There's some trap set somewhere (haven't checked) that makes downloaders produce a bad file.

Thanks for your reply
I just worked out how to grab the item..2 questions

1: how did you find "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/420/index.m3u8"
2: what are dev tools?

As i said, i need everything explaining simply as my stroke addled brain has difficulty just grasping stuff

Paul

[lomero]

about first question wait reply from white_snake

about second question mouse right click - Inspect is your answer (and look on network tab)

@ IceM
ok for _M3 mistake, but your shot show 20230412 version ... explain this one

[deeppurple1]

about first question wait reply from white_snake

about second question mouse right click - Inspect is your answer (and look on network tab)

@ IceM
ok for _M3 mistake, but your shot show 20230412 version ... explain this one

Thank you for your answer, i've never done anything like this before. I have worked out how to grab stuff from other sites using stream detector and ffmpeg But not from the site i want to!!

[white_snake]

Thanks for your reply
I just worked out how to grab the item..2 questions

1: how did you find "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/420/index.m3u8"
2: what are dev tools?

As i said, i need everything explaining simply as my stroke addled brain has difficulty just grasping stuff

Paul

Search for "index.m3u8" (Ctrl + F). You'll find the real .m3u8 is hidden inside the response to:

https://wcps.t-online.de/cmrs/magentamusic/media/v1/hlsstreaming/movie/9208205928595440217/9221438342941946642

Copy the part in bold:

src="https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true">

and put it in your N_m3u8DL-RE (or yt-dlp command):

N_m3u8DL-RE https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8

This link is slightly different (actually better) from the one I posted before, being a master playlist with all the video qualities available. When you run this command, you'll be presented with a list of all the available stream qualities to choose from.


[Attachment 77496 - Click to enlarge]

[2nHxWW6GkN1l916N3ayz8HQoi]

Alright. So according to this post:
https://forum.videohelp.com/threads/413704-Hello%21%21#post2726776

You are kinda old and also have some unfortunate medical issues on top of that. I'll give you the benefit of the doubt so I'll be understanding and offer you an easier solution. I know that learning new things can be a challenge at your age, especially tech stuff.

The next script gives you all the commands needed to download a video from https://www.magentamusik.de

import json
import re
import sys
import urllib.parse

import requests

if len(sys.argv) < 2:
    print("Usage: python script.py SOURCE_URL")
    sys.exit(1)

SOURCE_URL = sys.argv[1]
ASSET_URL = "{base_url}/cvss/magentamusic/vodclient/v2/assetdetails/{uri_id}/{asset_id}"
CMD_COMMANDS = \
    "ffmpeg.exe -i \"{download_url}\" -c copy output.mp4\nCOMMAND 1----\n" \
    "yt-dlp.exe \"{download_url}\"\nCOMMAND 2----\n" \
    "streamlink \"{download_url}\" best -o output.mp4\nCOMMAND 3----\n" \
    "N_m3u8DL-RE.exe \"{download_url}\" -sv best -sa best -ss all -M format=mkv\nCOMMAND 4----\n"

response = requests.get(SOURCE_URL)
href_values = re.findall(r'<link rel="preconnect" href="([^"]+)"', response.text)

base_url = [h for h in href_values if "online" in h][0]
asset_id = re.findall(r'"assetId":"([^"]+)"', response.text)[0]

uris = re.findall(r'"uri":"([^"]+)"', response.text)
uri_id = [u for u in uris if "http" in u][0]
uri_id = urllib.parse.unquote(uri_id[uri_id.find("http"):])
uri_id = re.findall(r'/seriesdetails/([^/]+)', uri_id)[0]

response = requests.get(ASSET_URL.format(base_url=base_url, uri_id=uri_id, asset_id=asset_id))
stream_urls = []
for info in json.loads(response.content.decode())["content"]["partnerInformation"]:
    for feature in info["features"]:
        stream_urls += [feature["player"]["href"]]
stream_url = stream_urls[0]

response = requests.get(stream_url)
hls_urls = []
for representation in json.loads(response.content.decode())["content"]["feature"]["representations"]:
    for package in representation["contentPackages"]:
        hls_urls += [package["media"]["href"]]
hls_url = hls_urls[0]

response = requests.get(hls_url)
download_url = re.findall(r'src="([^"]+)"', response.content.decode())[0]
print(CMD_COMMANDS.format(download_url=download_url))

It offers you 4 tools that you can pick from: FFmpeg, yt-dlp, streamlink, and N_m3u8DL-RE. Keep in mind, for certain videos, some tools may not work. I've seen that N_m3u8DL-RE fails for some while yt-dlp works without a problem. And also keep in mind that some tools aren't suitable for livestream recording. (As @snake already said).

then feed it to your favorite playlist downloader, be it yt-dlp, N_m3u8DL-RE or streamlink (the latter is the one best suited for live video, supposing it's not encrypted).

But if FFmpeg is the only tool you know and can deal with, I offered you the FFmpeg command as well. It is very slow when compared to the other tools, but it gets the job done at least. The script works very well for existing videos.

Example 1: https://www.magentamusik.de/iron-maiden-woa-konzert-vod
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

Example 2: https://www.magentamusik.de/angus-mcsix-woa-2023-9208205928595409182
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

Example 3: https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

I checked after downloading and is the best possible quality.


[Attachment 77497 - Click to enlarge]

If you don't know how to run the script... I've noticed in your screenshots that you opened the CMD and pasted an FFmpeg command. So I assume you have the basic skills to do that at least, which is good. First, open CMD and write: python --version

If it displays a specific version of Python then it is good. If instead you get this:

...is not recognized as an internal or external command...

Then Python is not installed so go here: https://www.python.org/downloads/windows/ and download an installer for Windows. Then open that installer and follow the steps displayed there. It has an interface so it should be fairly easy. In the end, run again that command: python --version, and see if it works.

If all is good you can run the script now. First check if you can view actual file extensions (Go to File Explorer, and see if "file name extensions" is checked, you can read more on Windows support dedicated forums). Then go into a folder and create an empty text file (with the .txt extension). Then open that file and drop that script I've written. Then rename that file to script.py (notice the .txt extension changes to .py).

Now open CMD in that directory and run the command:

python script.py URL

Where you replace the URL with your wanted video. Examples:

python script.py https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
python script.py https://www.magentamusik.de/angus-mcsix-woa-2023-9208205928595409182
python script.py https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
...etc...

Keep in mind, that the script works for existing videos. In theory, it should work for live streams as well. I doubt they modified their API for live streams to be different. But I never tested it for those so you never know. If you can provide a link to an actual live stream from that site, I could check. (live stream means somewhere the word LIVE should be displayed in big red letters). It doesn't matter if that is the live stream you want downloaded or not, just give one sample.

[deeppurple1]

Alright. So according to this post:
https://forum.videohelp.com/threads/413704-Hello%21%21#post2726776

You are kinda old and also have some unfortunate medical issues on top of that. I'll give you the benefit of the doubt so I'll be understanding and offer you an easier solution. I know that learning new things can be a challenge at your age, especially tech stuff.

The next script gives you all the commands needed to download a video from https://www.magentamusik.de

import json
import re
import sys
import urllib.parse

import requests

if len(sys.argv) < 2:
    print("Usage: python script.py SOURCE_URL")
    sys.exit(1)

SOURCE_URL = sys.argv[1]
ASSET_URL = "{base_url}/cvss/magentamusic/vodclient/v2/assetdetails/{uri_id}/{asset_id}"
CMD_COMMANDS = \
    "ffmpeg.exe -i \"{download_url}\" -c copy output.mp4\nCOMMAND 1----\n" \
    "yt-dlp.exe \"{download_url}\"\nCOMMAND 2----\n" \
    "streamlink \"{download_url}\" best -o output.mp4\nCOMMAND 3----\n" \
    "N_m3u8DL-RE.exe \"{download_url}\" -sv best -sa best -ss all -M format=mkv\nCOMMAND 4----\n"

response = requests.get(SOURCE_URL)
href_values = re.findall(r'<link rel="preconnect" href="([^"]+)"', response.text)

base_url = [h for h in href_values if "online" in h][0]
asset_id = re.findall(r'"assetId":"([^"]+)"', response.text)[0]

uris = re.findall(r'"uri":"([^"]+)"', response.text)
uri_id = [u for u in uris if "http" in u][0]
uri_id = urllib.parse.unquote(uri_id[uri_id.find("http"):])
uri_id = re.findall(r'/seriesdetails/([^/]+)', uri_id)[0]

response = requests.get(ASSET_URL.format(base_url=base_url, uri_id=uri_id, asset_id=asset_id))
stream_urls = []
for info in json.loads(response.content.decode())["content"]["partnerInformation"]:
    for feature in info["features"]:
        stream_urls += [feature["player"]["href"]]
stream_url = stream_urls[0]

response = requests.get(stream_url)
hls_urls = []
for representation in json.loads(response.content.decode())["content"]["feature"]["representations"]:
    for package in representation["contentPackages"]:
        hls_urls += [package["media"]["href"]]
hls_url = hls_urls[0]

response = requests.get(hls_url)
download_url = re.findall(r'src="([^"]+)"', response.content.decode())[0]
print(CMD_COMMANDS.format(download_url=download_url))

It offers you 4 tools that you can pick from: FFmpeg, yt-dlp, streamlink, and N_m3u8DL-RE. Keep in mind, for certain videos, some tools may not work. I've seen that N_m3u8DL-RE fails for some while yt-dlp works without a problem. And also keep in mind that some tools aren't suitable for livestream recording. (As @snake already said).

then feed it to your favorite playlist downloader, be it yt-dlp, N_m3u8DL-RE or streamlink (the latter is the one best suited for live video, supposing it's not encrypted).

But if FFmpeg is the only tool you know and can deal with, I offered you the FFmpeg command as well. It is very slow when compared to the other tools, but it gets the job done at least. The script works very well for existing videos.

Example 1: https://www.magentamusik.de/iron-maiden-woa-konzert-vod
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_Maiden2/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

Example 2: https://www.magentamusik.de/angus-mcsix-woa-2023-9208205928595409182
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_AngusMcSix/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

Example 3: https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
Output:

ffmpeg.exe -i "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" -c copy output.mp4
COMMAND 1----
yt-dlp.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true"
COMMAND 2----
streamlink "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" best -o output.mp4
COMMAND 3----
N_m3u8DL-RE.exe "https://streaming.magentamusik.de/vod/WW/WOA2023_JBO/hd/index.m3u8?yospace=true" -sv best -sa best -ss all -M format=mkv
COMMAND 4----

I checked after downloading and is the best possible quality.


[Attachment 77497 - Click to enlarge]

If you don't know how to run the script... I've noticed in your screenshots that you opened the CMD and pasted an FFmpeg command. So I assume you have the basic skills to do that at least, which is good. First, open CMD and write: python --version

If it displays a specific version of Python then it is good. If instead you get this:

...is not recognized as an internal or external command...

Then Python is not installed so go here: https://www.python.org/downloads/windows/ and download an installer for Windows. Then open that installer and follow the steps displayed there. It has an interface so it should be fairly easy. In the end, run again that command: python --version, and see if it works.

If all is good you can run the script now. First check if you can view actual file extensions (Go to File Explorer, and see if "file name extensions" is checked, you can read more on Windows support dedicated forums). Then go into a folder and create an empty text file (with the .txt extension). Then open that file and drop that script I've written. Then rename that empty file to script.py (notice the .txt extension changes to .py).

Now open CMD in that directory and run the command:

python script.py URL

Where you replace the URL with your wanted video. Examples:

python script.py https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
python script.py https://www.magentamusik.de/angus-mcsix-woa-2023-9208205928595409182
python script.py https://www.magentamusik.de/jbo-fruehschoppen-show-woa-2023-9208205928595409224
...etc...

Keep in mind, that the script works for existing videos. In theory, it should work for live streams as well. I doubt they modified their API for live streams to be different. But I never tested it for those so you never know. If you can provide a link to an actual live stream from that site, I could check. (live stream means somewhere the word LIVE should be displayed in big red letters). It doesn't matter if that is the live stream you want downloaded or not, just give one sample.

Hi
you are right, i am old and I do have issues. You've given me so much information here i thank you..I will need to work out how to use it haha..

I thought it was a simple matter to do what i want, It maybe to some but not to me. Life is a learning curve. Unfortunately there are quick people and there are slow, I'm one of the slow ones!..I will get there, i may ask what sounds like stupid questions to some, but to me, it maybe what holds me back

Paul

[2nHxWW6GkN1l916N3ayz8HQoi]

You've given me so much information here i thank you..I will need to work out how to use it haha..

I thought it was a simple matter to do what i want, It maybe to some but not to me

You're welcome. It is a simple matter once you install Python. After all, you only have to do this stuff once. I could give you an executable of that script that can be directly executed, but if I were you I wouldn't trust exe files from random new forum accounts. So I gave you the source code where you can actually see what you run. If you find this solution harder then you can do it manually as the other users have already suggested and showed you how.

PS: You can edit and shorten quoted comments when you write a message to avoid walls of text that are repeated.

[IceM]

Alright. So according to this post:
https://forum.videohelp.com/threads/413704-Hello%21%21#post2726776

The next script gives you all the commands needed to download a video from https://www.magentamusik.de

I'm always impressed by your skills to provide scripts

[IceM]

@ IceM
ok for _M3 mistake, but your shot show 20230412 version ... explain this one

It's an old beta (0.1.6)

N_m3u8DL-RE.exe --version
0.1.6

N_m3u8DL-RE.exe
Required argument missing for command: 'N_m3u8DL-RE'.

Description:
  N_m3u8DL-RE (Beta version) 20230412

[lomero]

oh yes, right. my mistake. so update to latest 0.2 beta ver. 20230628

[2nHxWW6GkN1l916N3ayz8HQoi]

I'm always impressed by your skills to provide scripts

Thanks for the nice words @ice To be fair, it's pretty doable to write a Python script for a data scraper on a specific website. All it takes is following a methodology and you'll be able to handle most sites. I'll try and explain the line of reasoning in the hope that it may help other users in writing their script alone:

Step 1. Let's say you found a URL (or a specific part of the URL, I'll explain later) that you want to figure out how to get programmatically in a Python script. That will be your target. For example, a URL pointing to an index.m3u8. First, go look into the source code of the origin page. For a short percentage of sites but still a respectable amount, you may find what you're looking for right on the source page. If not proceed.

If you reach step 1 again and your target URL is the source page, you can stop since you solved the problem and you have the entire chain of requests that you can replicate programmatically.

Step 2. Look for all the previous requests right before the target request. If you find your target in one of these requests, that request will be your new target, and go back to step 1. If you didn't find anything useful, proceed.

Step 3. If your target is an actual URL, you need to split that URL into smaller problems. For example:
https://base_url/path1/path2/id1/id2?p1=v1&p2=v2 may be split into base_url, path1, path2, id1 and id2, p1, v1, p2, v2

For each one of these components, you need to consider it a different target and go back to step 1. If some components seem to be constant and not unique (like an id), then you can skip them. For example, a path to an endpoint from the API may be hardcoded in the script. In some cases, however, you need to also be able to deduce the base_url because of regional problems if you want your script to be used by the wide public. For example: AWS regional URLs may be dependent on the continent, etc.

You may try to skip the query parameters however, in some cases, you may need to check even them. In some tougher cases, you need to consider even specific headers as different components that you need to look for.

If your target is an actual separated component then you hit a dead-end. Go back to step 2 and look more carefully. If you still reach here, proceed.

Step 4. You might need to look for additional requests that may obtain your target component. Like, let's say, a call to an endpoint that returns a token and is never even called in the source page. It is worth checking other pages separated from your source page to obtain additional information. If even this doesn't help, then it's game over.

I hope this summary helps someone in writing their data scraper script. In short, it is the classic Divide et Impera. Divide a bigger problem into smaller ones. The best part is you don't even need to read a bunch of documentation related to their API since you only focus on specific requests that you need by monitoring them.

Edit: I made a more detailed guide here
https://forum.videohelp.com/threads/415366-the-slacker-s-guide-to-mass-downloading-(mo...on#post2743977

[white_snake]

@OP: FYI, they just added an extractor for Magentamusik in the latest version of yt-dlp. Usage:

yt-dlp https://www.magentamusik.de/iron-maiden-woa-konzert-vod

[deeppurple1]

Sorry I have not been here for a while, I've had another Stroke and have been in hospital for a few weeks..

I'd like to say a huge THANK you for all your replies and advice...Unfortunately, i'm going to have to completely re-learn and go through everything again...Hopefully i can do that before the Rock Hard Festival starts to livestream on the 17th May

[white_snake]

Sorry I have not been here for a while, I've had another Stroke and have been in hospital for a few weeks..

I'd like to say a huge THANK you for all your replies and advice...Unfortunately, i'm going to have to completely re-learn and go through everything again...Hopefully i can do that before the Rock Hard Festival starts to livestream on the 17th May

Hope you're feeling better now, mate.

Anyway, if you have troubles getting it to work, just make sure to post here on May 17 and somebody should be able to help you almost immediately.